Papers with mutual information maximization
Prompt-based Logical Semantics Enhancement for Implicit Discourse Relation Recognition (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for identifying discourse relations without explicit connectives are limited by the availability of annotated data. |
| Approach: | They propose a method that injects knowledge relevant to discourse relation into pre-trained language models through prompt-based connective prediction. |
| Outcome: | The proposed method achieves outstanding performance against the current state-of-the-art models. |
ThinkAnswer Loss: Balancing Semantic Similarity and Exact Matching for LLM Reasoning Enhancement (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for knowledge distillation use Chain-of-Thought (CoT) and answer pairs, but they lack appropriate supervision signals. |
| Approach: | They propose a framework that decouples CoT and answer supervision . the framework applies semantic similarity constraints while maintaining strict literal matching for the answer . |
| Outcome: | The proposed framework decouples CoT and answer supervision while maintaining strict literal matching for the answer. |
NEXUS Network: Connecting the Preceding and the Following in Dialogue Generation (D18-1)
Copied to clipboard
| Challenge: | Sequence-to-Sequence models favor short generic responses . however, the model is not suitable for modeling dialogues . |
| Approach: | They propose a model that connects preceding and following conversations to a prior distribution to avoid non-differentiability of discrete natural language tokens. |
| Outcome: | The proposed model is highly efficient in learning the backbone of human-computer communications, but favors short generic responses. |
InfoCL: Alleviating Catastrophic Forgetting in Continual Text Classification from An Information Theoretic Perspective (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have identified the severe performance decrease on analogous classes as a key factor for catastrophic forgetting. |
| Approach: | They propose a replay-based continual text classification method that uses fast-slow and current-past contrastive learning to perform mutual information maximization and better recover previously learned representations. |
| Outcome: | The proposed method achieves state-of-the-art on three text classification tasks. |
A Mutual Information Perspective on Knowledge Graph Embedding (2025.acl-long)
Copied to clipboard
| Challenge: | Existing knowledge graph embedding techniques suffer from high intra-group similarity, loss of semantic information, and insufficient inference capability, particularly in complex relation patterns such as 1-N and N-1 relations. |
| Approach: | They propose a knowledge graph embedding framework that leverages mutual information maximization to improve the semantic representation of entities and relations. |
| Outcome: | Extensive experiments on benchmark datasets demonstrate the effectiveness of the proposed method, with consistent performance improvements across various baseline models. |